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NOTE: 



Reg. No.: 37,424 



Tel. No.: ( 734) 913-9300 




(type or pnnt name of attorney) 

280 N. Old Woodward Ave, 

P.O. Address 0 . , . ~ rt 
Suite 400 

Birmingham, MI 48009 



(Reissue Application Transmittal [17-1]— page 4 of 4) 



VR-28-1999 09:48 GIFRMh KRRSS r'.kW 



Practitioner's Docket Ho. FNI-01503/03 

REI88U£ APPLICATION BY ASSIGNEE, OFFER TO SURRENDER 

(37 C.F.R §1.17S) 

To the Assistant Commissioner for Patents : 

The undersigned makes this statement as part of the accompanying 
reissue application for the reissue of Letters Patent No. 5,625,410 
| for an improvement in VIDEO MONITORING AMD CONFERENCING SYSTEM, 
| granted April 29, 1997 to Kinya Washino and Barry Schwab. The 
1 patent, assigned to Kinya Washino, is the owner of the entire 
| interest in eaid original patent and hereby otters to surrender said 

S Letters Patent, 
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| Filed herewith is a request for abstract of title, and payment 

of the fee required by 37 C.F.R. 11.19(b) {4) as required in rei9sue 
applications. ^ 



8 



Date: ftpnl 36, 1939, 

Kin 



nJaWashirif 



2 



flPR-28-1999 09:57 



G1FF0RD. KRflSS P - 02 



CERTIFICATE UNDER 37 CFR 3.73(b) 

Inventor: Kinya Washino 

Patent No.: 5,625,410 Issued: April 29, 1997 

For: VIDEO MONITORING AND CONFERENCING SYSTEM 

jCinyaWMhino T a n individual- c ertifies that he is the assignee of the entire right, title and 
interest in die patent identified above by virtue of either: 

A. FX} An assignment from the inventor of the patent identified above. The assignment was 

recorded in the Patent awl Trademark Office on February 7, 1997 at Reel 8339, Frame 
0123. 

B, ( ] A chain of tide from the inventor(s), of the patent application identified above, to the 

current assignee as shown below: 

1. From: To: 

The document was recorded in the Patent and Trademark Office on at Reel 

f Frame , or for which a copy thereof is attached, 

2. From: To: 

The document was recorded in the Patent and Trademark Office on at Reel 

t Frame ♦ or for which a copy thereof is attached. 

The undersigned has reviewed all the documents in the chain of title of the patent identified above 
and, to the best of the undersigned's knowledge and belief, title is in the assignee identified above. 

The undersigned (whose title is supplied beiow) is empowered to act on behalf of the assignee. 

I hereby declare mat all statements made herein of my own knowledge are true and that all 
„^;..L ^3de on information and belief are believed to be true; and further that these 
statements were made with the knowledge that willful fake statements and the like so made are 
;„ « LsjxSamM, or bom, under Section 1001 of Title 18 of the Oiuted Ss&es 
code, and mat such willfui false statements may jeopardize the validity of the application or any 
patent issued thereon. 



Date: Anrit7i> 1999 

— 1 Kinya Washino 
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IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 



In re application of: Washino et al 
Serial No. : Group No. 

F j_]_ ec l : Examiner : 

z For: VIDEO MONITORING AND CONFERENCING SYSTEM 



< 
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PRELIMINARY AMENDMENT 



Assistant Commissioner for Patents 
1 Washington, D.C. 20231 



Dear Sir: 

Prior to examination, please amend the above-referenced 
I reissue application as follows: 



§ IN THE SPECIFICATION 



Column 5, line 44, replace "as a the utility" with --as a 



utility - 



Column 6, line 2 6 , after "may" insert --be--. 
§ Column 8, line 3, after "interconnection" delete the comma 



* (,) . 

o 

g= Column 9, line 21, replace "he" with --be--. 

Q 

< Column 10, line 7, delete "preferably" (first occurrence) . 

lu 



£ Respect fullY^ubmitted, 
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CD 
CO 

% By 



£ John G.//Posa 

g Reg. Ncf/ 3 7,424 

2 Gifforfl/, Krass, Groh, Sprinkle, 

5 Anderson & Citkowski 

280 N. Old Woodward Ave., Ste 400 
Birmingham, MI 48009 
Date* Apr. 28. 1999 (734) 913-9300 FAX (734) 913-6007 
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VIDEO MONITORING AND 
CONFERENCING SYSTEM 

REFERENCE TO RELATED APPLICATIONS 

This application is a continuation-in-part of U.S. Ser. No. 
08/050.861. filed Apr. 21, 1993 now U.S. Pat. No. 5.450.140 
and Ser. No. 08/298.104. filed Aug. 30. 1994 now U.S. Pat. 
No. 5.537.157. 

FIELD OF THE INVENTION 

This invention relates generally to video monitoring, and, 
more particularly, to such systems employing means for 
digitizing camera images for display at a first image size, 
sampling rate, and frame rate, and for digital storage at a 
second image size, sampling rate, and frame rate. 

BACKGROUND OF THE INVENTION 

Existing systems for monitoring video signals from mul- 
tiple sources either have relied upon switching means to 
sequence through each of the sources in a predetermined 
pattern, or else implement some form of "split screen" 
display, in which a conventional video monitor displays, for 
example, the output of four cameras, each occupying a 
portion of the display screen. Often, the video output is also 
suppiied to signal storage means such as a VCR which is 
operated in a single -frame recording mode, providing a 
time- lapse record of the camera outputs. These types of 
systems are in common usage, and are well known in the art 

More recently, the availability of digital techniques for 
video signal processing and data compression has opened 
new video monitoring alternatives. Mathisen. for example. 
U.S. Pat. No. 4.198.656. describes a system employing a 
signal selector which is controlled by signal storage means, 
including a VCR functioning as a time-lapse frame recorder, 
and disc storage means functioning as a frame -store type 
device which provides continuity in the display while the 
selector is advancing through its sequence to the next signal 
source, or to the next recorded signal from one particular 
source. Each image is further provided with a digital code, 
which enables the image source to be identified upon 
playback. 

Katz (U.S. Pat. No. 5.216.502) discloses a system for 
monitoring and recording transactions at multiple cashier 
lanes. According to the preferred embodiment, output sig- 
nals from four cameras are fed to a four-quadrant 
multiplexer, which contains a frame store having provisions 
for reducing each camera image to one-quarter size and then 
displaying it in one of the four quadrants of the video 
monitor. The combined output signal is then recorded on a 
Video Cassette Recorder (VCR) to provide a more perma- 
nent record of the transaction. No description is given of the 
size -reduction method, but this kind of display may be 
implemented by employing a ame -compression scheme for 
the horizontal Lines, and eliminating alternate scan lines 
from the vertical dimension of the video frame. Data from 
che individual cash registers is encoded digitally, and then 
either recorded on lines of video in the vertical- interval or 
else recorded on the audio tracks of the VCR. The moni- 
toring provisions include a video overlay generator, so that 
transaction information may be displayed concurrendy with 
images of the event. Provisions are also included for "selec- 
tive recording of representative frames of video, as triggered 
by transaction events at individual cashier lanes. This further 
reduces the number of frames recorded, and extends the 
recording duration of the video cassette. 
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Neither of these two inventions disclose the use of image 
data compression schemes, nor of image resizing by digital 
means. Katz does disclose selective recording of frames of 
video, but only in analog form, and using conventional video 
5 recording means, such as a VCR. 

Blum et al. (U.S. Pat. No. 5237,408) disclose a system 
employing multiple interface circuit boards containing pro- 
visions for capturing and digitizing images from a plurality 
of cameras, further storing a slow- scan sequence of images 

10 for each camera in active buffer memory means as RAM on 
each of the interface boards. The display of these image 
sequences is preferably controlled by a PC with disk storage 
capabilities, and individual images selectively may be 
recorded on the hard disk, under operator control. Storage 

15 capacity is limited, however, because no image data com- 
pression is provided, and it is not practical to store large 
quantities or sequences of images. There are no provisions 
for automatic recording of images, nor any provision for 
storing these images in a data form differing from the display 

20 format image size. 

Gormley (U.S. Pat No. 5258.837) discloses video com- 
pression means as a method for fining multiple camera 
images within a single video display screen. However. 

25 Gormley clearly indicates that the system relies on fixed 
image-size compression of the digitized kcomiag video 
signal, and does not perform any kind of bandwidth com- 
pression. Provisions are included for storing images on a 
video recorder, but. as in other systems, these images are 
stored by taking the screen display as a whole, rather than 
storing independently constituted representations of the 
individual camera outputs. 

In general, these conventional systems, regardless of the 
subsequent signal processing employed, accept analog video 

35 signals from the camera sources for input to the monitoring 
system. As such, these signals are susceptible to distur- 
bances such as RF interference, ground-loop noise, and 
high-frequency signal loss due to long runs of coaxial 
cables. Automatic video switcher units sequence the source 

^ signals at a relatively slow rate, requiring as much as 16 
seconds delay between successive viewings of a particular 
source when using a one source-per-second rate to sequence 
through 16 sources. When such a system is equipped with a 
time -lapse video recorder, the recording medium is 
45 expensive, has low resc^ungo. and is limited in recording 
capacity. " * ~ ? 

Other systems, such as video conferencing arrangements, 
use multiple camera images and audio sources which are 
coordinated through electronic switching means and image 

50 display means, with interconnections between conferencing 
rooms locally or at remote sites via telephone lines or other 
communicarioQS links. Fabris et al. (U.S. Pat. No. 4 J 16. 
156) discloses a sophisticated implementation of such a 
system, employing satellite links for long-distance 

55 communications, a full range of camera remote controls 
(including lens controls and pan- tilt-head controls), and 
touch-screen monitor facilities for system controls of cam- 
eras and signal switching. This system, however, has no 
provisions for image storage, or any application of image 

5q data compressiou means coupled with display means or 
image storage means. 

SUMMARY OF THE INVENTION 

The present invention implements an automated video 
65 monitoring system by way of a PC-based platform employ- 
ing display windowing software, with camera sources being 
interfaced to an input circuit board which includes provi- 
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sions for image data compression. Using a basic image size 
in pixels of 320x240. and optionally including color pro- 
cessing employing a Y-tl-V 4:1:1 or 4:2:2 sampling 
technique, a range of performance standards are established. 
An economical simultaneous display of 4 sources in a 2x2 
configuration on a conventional I0 n VGA-format (640x480 
pixels) monitor may be upgraded to a more elaborate 
24-source (plus one utility window, which also may be used 
as a graphical-based input source device for transmitting 
control commands to the individual cameras) display in a 
5x5 configuration on a high-resolution 20" (1600x1200 
pixels) monitor. Other combinations are possible, including 
arrays of images of size 640x480 pixels or even 800x600 
pixels, depending on the screen size of the display monitor 
and the capabilities of the video display adapter circuit card. 
In addition, not all image windows need to be of the same 
size, nor updated at the same rate, but rather they may be 
mixed and combined based on particular applications. 
Remote controls for the individual cameras may be imple- 
mented by way of windowing software andVor monitor 
touch-screen devices. Automatic sensing of particular events 
(representing security alarms, equipment or process 
disturbances, or a change in the person speaking in a 
videoconferencing environment) may be employed to cause 
reconfiguration by way of resizing the image or modifying 
the update rate of individual windows on the display screen, 
or by modifying the data format of recording images on a 
storage device. 

Storage of images may be implemented by way of a tape 
back-up device, such as a DAT or 8- mm tape recorder, which 
are capable of storing as much as 960 hours of monitoring 
images, or by way of disk storage devices, preferably 
including removable disk drives such as magneto-optical 
disks or PCMCIA-compatible disk-drive modules. Images 
are preferably stored as a succession of data-compressed 
representations, corresponding to various window sizes 
sampled at diverse update rates: however, though the image 
representations need not be identical to the sizes and rates 
used for video monitors displaying the various images. In an 
alternative embodiment, cameras are provided with data 
compression means at the camera location, with a 
bi-directional data link providing control signals from the 
PC. and accepting data-compressed images from the cam- 
eras. Depending on the data rates selected, the transmission 
of these signals may be by means of coaxial cables, twisted- 
pair lines, or fiber-optic cables. These signals may alterna- 
tively be o-ansmioed over internal PBX telephone lines, or 
over greater distances, including satellite links, as would be 
the case for remote monitoring of facilities or videoconfer- 
encing. The use of digital images allows the application of 
various useful techniques, such as digital noise reduction 
and motion-sensing, to trigger security alarms, for example. 

It is an object of the invention to provide a more efficient 
method for monitoring camera outputs by means of a 
multiple- window display system implemented on a com- 
puter platform. 

It is another object of the invention to provide an 
improved recording system for event-logging or security 
applications, by means of storing data-compressed digital 
images, including identifying information, which are more 
representative of events than typical analog time-lapse 
recordings, and further to implement this system on a 
computer platform. 

It is a further object of the invention to provide extended 
recording time for event-logging or security applications by 
means of either tape -based, or disk-based, or both tape- 
based and disk- based data recording means. 
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It is yet another object of the invention to provide a 
system for remote monitoring of cameras by means of 
transmission of data-compressed digital images over com- 
munication links. 

5 It is yet a further object of the invention to provide a 
convenient method of implementing videoconferencing 
facilities by means of a PC- based platform. 

BRIEF DESCRIPTION OF THE DRAWINGS 

10 

FIGS. 1-6 show various possible screen display configu- 
rations according to the invention; 

FIG. 7 is a functional diagram of a PC-based system 
wherein plug- in printed-circuit boards implement digital 
15 processing for analog input signals; 

FIG. 8 is a functional diagram of a plug- in printed-circuit 
board introduced with reference to FIG. 7; 

FIG. 9 is a functional diagram of a universal camera 
adapter having digital output signals; 

FIG. 10 is a diagram of an integrated camera system: 

FIG. 11 is a functional diagram of a PC-based monitoring 
system adapted for videoconferencing at multiple remote 
locations; 

25 FIG. 12 is an overhead view of a physical layout of the 
system of FIG. 11; 

FIGS. 13 A and 13B are drawings of a multiple -camera 
monitoring unit for table-top use in videoconferencing; 
FIG. 14 is a side-view of an expanded muitipie-camera 
^° monitoring unit implemented for videoconferencing; 

FIG. 15 is a table listing a variety of possible operating 
modes, depending on computer monitor display facilities, in 
either a monitoring or a videoconferencing version of the 
35 invention. 

DETAILED DESCRIPTION OF THE 
PREFERRED EMBODIMENT 

The present invention implements an automated video 

40 monitoring system by way of a PC-based platform employ- 
ing display windowing software, with camera sources being 
interfaced to an input circuit board which includes provi- 
sions for image data compression. The basic video window 
size of 320x240 pixels from each camera source can be 

45 displayed on a variety of video monitors, in a number of 
formats, depending on system complexity. The preferred 
recording medium is a 4- mm helical-scan far* cartridge, 
commonly referred to as a digital audio tape (DAT). Each 
tape cartridge is capable of storing 10 GB (gigabytes) of 

50 data. All recording times in the explanation below are based 
a data-compression ratio of 100:1. utilizing a 4:2:2 Y/U/V 
sampling method for color images. Other higher capacity 
media, such as 8-mm tapes capable of 20 GB of darn storage, 
may be employed when longer times are desired 

55 FIG. 15 shows a variety of possible operating modes, 
depending on the particular implementation of the PC-based 
monitoring system- These six representative modes will be 
explained in detail with the understanding that the invention 
is by no means limited to the specific examples shown, and 

6o thai many different alternatives are possible. It should be 
noted also that in certain of the implementations shown, the 
enure screen may not be occupied by the various windows, 
and any unused areas at the top. bottom, or sides may be 
utilized to display warning messages, control buttons, or 

65 other such facilities. 

FIG. 1 shows a computer monitor display for a system 
configured in accordance with Mode I as listed in FIG. 15. 
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Using a commonly available 14" VGA-format computer 
monitor with a dimension in pixels of 640x480. four win- 
dows having a dimension in pixels of 320x240 may be 
displayed simultaneously Recording this data at the rate of 
one frame per second (1 fps; in black: and white (B/W) and 
with 100:1 data-compression on a 10 GB DAT tape provides 
approximately 960 hours of images per tape, or alternatively 
more than 480 hours of color images. A second monitor 
optionally may be used to display utility windows including 
various system operating controls, as, for example, camera 
pan. tilt. zoom, focus, and so forth. 

FIG. 2 shows a computer monitor display for a system 
configured in accordance with Mode II as listed in FIG. 15. 
Using a 15" to 17" SVGA- format computer monitor with a 
dimension in pixels of 1024x768. nine windows, each 
having a dimension in pixels of 320x240. may be displayed 
simultaneously. Eight windows may be implemented as 
camera displays, and the ninth window may be used for the 
continuous display of the utility window, as described 
above. Recording of all eight windows at 1 fps will allow 
480 hours of B/W images in this configuration per 10 GB 
DAT tape, or alternatively more than 240 hours of color 
images. 

FIG. 3 shows a computer monitor display for a system 
configured in accordance with Mode IH as listed in FIG. 15. 
Using a 17" to 20" high-resolution computer monitor with a 
dimension in pixels of 1240x 1024, eight windows having a 
dimension in pixels of 320x240 may be displayed, in 
addition to two windows having a dimension in pixeis of 
640x480. One of these larger windows may be implemented 
as a utility window, and the other larger window may be 
implemented for sequential display of the eight camera 
inputs at full camera resolution having a dimension in pixeis 
of 640x480. Recording all eight camera image windows, 
and also the input- scan window, at 1 fps will allow 360 hours 
of B/W images in this configuration per 10 GB DAT tape, or 
alternatively more than 180 hours of images in color. 

FIG. 4 shows a computer monitor display for a system 
configured in accordance with Mode IV as listed in FIG, 15. 
Using a 20" high-resolution computer monitor with a dimen- 
sion in pixels of 1600x1200, 25 windows with a dimension 
in pixels of 320x240 may be displayed simultaneously. 
Twenty -four windows may be implemented as camera 
displays, and one is implemented as a the utility window. 
Recording all twenty-four windows, and also the utility 
window, at 1 fps will allow 180 hours of B/W images in this 
configuration per 10 GB DAT tape. If color recording is 
desired, a 20 GB 8-mm data cartridge will allow 180 hours 
of color images. 

FIG. 5 shows a computer monitor display for a system 
configured in accordance with Mode V as listed in FIG. 15. 
Using a 20" high-resolution computer monitor with a dimen- 
sion in pixeis of 1600x1200, twelve larger windows having 
a dimension in pixeis of 400x300 may be displayed, as well 
as one large high-resolution window, with a dimension in 
pixels of 800x600. to display the sequentially scanned 
output of the camera images. Recording in this format at 1 
fps will provide 180 hours of B/W images in this configu- 
ration per 10 GB DAT tape. If color recording is desired, a 
20 GB 8-mm data canndge will allow 180 hours of color 
images. 

FIG. 6 shows a computer monitor display for a system 
configured in accordance with Mode VI as listed in FIG. 15. 
Using a 14" to 17" SVGA computer monitor with a dimen- 
sion in pixels of 1240x1024. and implementing a smaller 
window size with a dimension in pixels of 240x180, eight 
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windows are available for the camera image displays, and 
two large windows of 480x360 are available for 
sequentially-scanned camera images. Recording in this 
screen format at 1 fps allows 480 hours of B/W images in 

5 this configuration on a 20 GB 8-mm data cartridge, and 
allows 240 hours of images in color. 

In any of the display configurations just described, it is 
also possible to use a video overlay technique to display the 
utility controls superimposed on the camera video. Regard- 

10 less of which display or recording format is used, the PC 
based monitoring system includes various facilities and 
features, which will now be described in further detail. 

In operation, the user need only observe the various 
display windows on the monitor screen, rather than concen- 

15 trate on many monitors simultaneously, thereby reducing the 
risk of missing an important event. Since no video switcher 
is used in recording, and images from all sources are 
continuously recorded at the selected frame rate for each 
source, more information is recorded than in conventional 
analog systems, wherein events may be missed due to the 
sequential switching of input images. The digital format also 
improves the picture quality, as the signai-to-noise ratio will 
be higher than for analog systems. In addition, there is no 
loss of quality during recording or playback, and because the 

25 recording technique is digital, other types of information 
optionally may recorded along with the camera data, such as 
audio, time, date, location, etc. 

An additional feature is the capability to implement a 

3Q dual-recording-media option. This facility provides the abil- 
ity to record simultaneously both on a tape (for high 
capacity, long term storage) and also on a removable media, 
such as a removable hard disk (i.e. PCMCIA) or magneto- 
optical disk (for short-term storage of up to 24 hours of 

35 images). These disks facilitate high-speed searching of 
recorded information without interrupting the tape, as well 
as providing a back up for the recording on the tape. In 
addition, the system is capable of simultaneously searching 
the recorded images on the disk storage unit while continu- 

^ ing to store images on the tape storage uniL 

It should be noted that there is no requirement that the 
image sizes or frame rates utilized for the video display 
match those utilized for the storage media. In practice, these 
two specifications may not agree, and will be determined by 

4.5 other factors, such as operator manipulation of the displayed 
image sizes or changes resulting from the detection of alarm 
conditions. 

FIG. 7 shows a functional diagram of an analog input- 
digital processing card installed in the PC which allows the 

50 use of existing analog cameras and cables with the PC-based 
monitoring system. This card, which may be obtained from 
such manufacturers as Nova. Model No. V-SW, features f our 
video inputs 2a-2d having balanced or differential input 
circuitry for good noise immunity (or optically-coupled 

55 inputs for liber-optic cabling), four video amplifiers 4a-4<l 
a 4x1 video switcher 6 by which any one of the four video 
inputs may be selected as a signal source, and an analog- 
to-digital (A/D) converter S. The recording of camera inputs 
is digital, however, there is no provision for remote control 

60 of camera functions such as pan. tilt. zoom, and so forth. If 
additional inputs are desired, multiple cards may be installed 
in the PC. The output of the A/D converter 3 is supplied to 
the graphics processor and image data-compression engine 
10. This unit performs the various functions required to 

65 configure the image sizes and frame rates as specified by the 
equipment operator. Because the overall data recording 
bandwidth of the system is a factor in the selection of the 
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individual picture data races, computer software is provided 
to implement menu-driven management of the data band- 
width allocation to the various image sources. Based on this 
coniigurauon, signals are provided to the video switcher 4 to 
facilitate the selection of the particular image sources as 
required The data-compressed images are provided in digi- 
tal form to the microprocessor data bus 12. The micropro- 
cessor unit (not shown) in turn provides control signals to 
the graphics processor 10 by way of this data bus. in 
accordance with the image data allocation configuration 
selected. 

FIG. 8 is a functional diagram of a digital input/output- 
digital processing card which implements bi-directional 
digital communications to and from the camera/adapter unit 
over a single transmission line, thereby greatly reducing the 
cost of cable and installation, and allowing existing wiring 
to be used. Various implementations of this printed-circuit 
card are required depending on the type of network inter- 
connection media selected In addition, system performance 
will depend on the type of networic interconnection media. 
As an example, a fiber optic network: will have a higher 
transmission rate and better signal quality than a telephone 
networic Interfacing technology for these communication 
methods are in common usage and well known in the art. 
This printed-circuit card also serves four cameras through 
inputs 52a-52d* and additional cards can be implemented if 
required The bi-directional digital video and camera control 
data switcher 54 serves to select the individual image 
sources as explained in reference to FIG. 7. These source 
signals are then provided to the graphics processor and 
image data-compression engine 56. which provides similar 
functions to those provided by the graphics processor 10. 
discussed in reference to FIG. 7. The processed data- 
compressed images are then provided to the microprocessor 
data bus 60. However, in this implementation, the 
bi-directional nature of the data switcher 54 is exploited to 
provide full functional control of the camera facilities 
througn the bi-direcaoaal controller 58. These functional 
controls include the cameras themselves (lens and exposure 
control) and the camera' s physical mounting provisions (pan 
and alt). The various control signals for these functions are 
those traditionally provided to these kinds of equipment, and 
these concepts are well known in the art. In an alternative 
embodiment, the individual cameras are each equipped with 
separate image data-compression facilities, utilizing such 
techniques as motion-JPEG or MPEG compression. In this 
case, control signals for these additional functions are pro- 
vided from the microprocessor bus 60 through the 
bi-directional controller 58 and the data switcher 54. 

FIG. 9 is a functional diagram of a digital output universal 
camera adapter. This is a fully digital version of the camera 
adapter as described in co- pending U.S. patent application 
Ser. No. 08/050.861. titled "PERSONAL-COMPUTER 
BASED VIDEO PRODUCTION SYSTEM" now U.S. PaL 
N'o. 5.450.140. The A/D-converter, digital image data- 
compressor, and bi- directional interface camera adapter 100 
accepts analog audio and video signals from the camera 102, 
and converts them to digital signals in anticipation of the 
transmission of these signals over the interconnection net- 
work 104. The camera adapter also receives camera control 
commands from the PC by means of the interconnection 
network, and translates them into the appropriate pan. tilt, 
zoom, focus and ins control signals for the particular camera 
equipment, including the camera lens 106 and the pan/tilt 
mounting facilities 108. In addition, the camera adapter also 
has inputs for several ''alarm system" type sensors 110. as. 
for ex ample . motion detectors. photoceE detectors, or sunple 
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switches. These alarm signals are digitized, encoded, and 
then transmitted to the main PC by means of the 
interconnection. network. Power is supplied for this equip- 
ment from a local source 112. This camera adapter is 

5 implemented to provide a full-function system, adaptable to 
ail existing types of cameras and control equipment. 
However, because of the iarge number of interconnections 
involved in the adapter, camera, pan/tilt unit, and sensors, 
the installation process may be somewhat complicated. 

10 By equipping an individual camera input with digital 
frame store capabilities, it is possible to detect an alarm 
condition based on deviations from the normal-state image 
scene. For exampie. if a camera is monitoring a door exit or 
an area of a warehouse aisle that is not utilized during 
evening hours, any change in the image would represent an 

15 alarm condition. To allow camera movement, this sensing 
process would be disabled during pan. scan. tilt, zoom, and 
other camera positioning controls, and the sensing would be 
re -enabled after camera positioning ceased, or by manual 
operator control. 

20 FIG. 10 is a functional diagram of an integrated camera 
system according to the invention including means to over- 
come problems associated with having a physically separate 
camera, pan/tilt unit, and adapter unit The integrated system 
includes a camera 150 with pan/tilt control. A/D converter, 

25 digital data-compression circuitry, and requisite interfacing 
circuitry. Such provisions are preferably included in the 
camera base structure 152, which simplifies installation, as 
no separate interconnecting cables are required. The inte- 
grated camera system optionally may include provisions for 

30 supplying electrical operating power from a network coaxial 
cable, and thereby may be implemented with only a single 
interconnecting cable. An integrated camera system of this 
type may be implemented in a physical package which is as 
small as the camera itseif. since most of the electronics may 

35 be highly miniaturized by developing custom integrated 
circuits, including LSL ASIC, DSP. and mixed- signal pro- 
cessing. 

It will be appreciated that in any of these 
implementations, alarm or sensor signals may be utilized to 

40 automatically re-configure the system operating mode. as. 
for example, increasing the frame rate or image size for an 
image source associated with the sensor which has initiated 
the alarm signal condition. As explained above, the dis- 
played windows and image sizes may be reconfigured into 

45 an operating mode different from the reconfiguration of the 
digital storage mode. If desired, the operator may choose to 
allow the system to automatically adjust the compression 
ratio utilized for a particular window in response to alarm 
signal conditions. .Alternatively, the compression ratio may 

50 be adjusted in response to the selection by the operator of a 
particular window for closer monitoring, by switching to an 
image window having larger dimensions in pixels. For some 
applications, the use of a resolution-independent data com- 
pression scheme, such as the 'Tractai Compression" method 

55 of Iterated Systems. Inc.. will be preferred, since images 
compressed by this method may be resized to fit larger or 
smaller windows as desired, without Loss of apparent reso- 
lution. In addition, automatic switching of the audio signals 
associated with a particular window m response to alarm 

60 signal condition will enable the operator to fully investigate 
such events, and the audio signals may serve to attract the 
operator's attention to the event. Such switching optionally 
may be coupled, with operator selections for monitoring 
purposes as well, and in both of these options may be 

65 integrated into the system previously described for recording 
the images, with or without any associated audio signals, 
onto the mass- storage media. 
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FIG. 11 illustrates a PC-based monitoring system imple- 
mented for videoconferencing at multiple remote locations. 
In this application of the invention, a main control center, 
shown generally as 200, is interconnected, by way of a 
network, with remote locations, shown generally as 204 to 
210, designated as "1" through **>T. The network 202 may 
be implemented by way of telephone lines (slow speed). 
ISDN lines (medium speed), or alternatively coaxial cable or 
fiber optics cables (high speed;. The higher speed imple- 
mentations permit transmission at higher frame rates and 
higher image resolutions. As an alternative, wireless net- 
works or microwave links optionally may be used. Network 
communication between the main and remote locations is 
bi-directional; the main computer receives audio, video, and 
sensor data from each of the remote locations, and transmits 
controls for camera pan. tilt, focus, zoom, and so forth. 
Audio and video communications for the system operators 
also is provided, by means of an additional camera and 
headset at each location. 

The application of these PC-based monitoring techniques 
also may he implemented in cases in which the central 
monitoring area may be located at some distance from the 
remote site. In this case, the physical hardware and network- 
ing facilities will follow the example shown in FIG. 11, with 
the understanding that "Location 1" through "Location IN™ 
will now refer to separate monitoring for security applica- 
tions or other monitoring purposes. 

The purpose of the main control center is to manage 
multiple cameras at multiple locations. By monitoring and 
controlling the remote locations from a central location 220, 
the system may function without an operator at each loca- 
tion. Optionally, additional computers shown, by way of 
example, as 222 and 224 at the main location may be 
implemented to simplify the monitoring and control of the 
remote locations. The main control center stores the data 
from remote locations in the main control storage facilities, 
shown, by way of example, as 238. 240 and 242. using 
digital tape and removable media, including, by way of 
example. PCMCIA-based removable media or magneto- 
optical (MO) media. The images may be stored at various 
frame rates and resolutions, depending on me requirements 
for the intended use of the stored images. Intercom provi- 
sions as shown as 230, 232 and 234, facilitate the coordi- 
nation of activities between the main control center and the 
various remote locations, and video camera units are pro- 
vided at each of the computers, shown, bv wav of example, 
as 244. 246 and 243. 

Each remote location includes a PC/monitor 250, with 
multiple video cameras, shown, by way of example, as 252 
and 254. including provisions 256 for image and data 
storage utilizing digital tape and other removable media. 
This remote location may be operated under control from 
either the main location (as described above), or a local 
operator, or by specific software designed for automatic 
control. In the case of software-controlled operations, pro- 
visions are included for control to be assumed by the main 
control center, if necessary. As in the case of the main 
control computers, each of the remote computers is 
equipped with intercom facilities as 258 and video cameras 
as 260. 

In the traditional videoconferencing situation, only one 
camera and monitor are used. Uofortunateiy. this results in 
an unnatural scene which provides the users with only a 
single viewing perspective. The use of a wide-angle camera 
leas which is needed to include many people seated around 
a table creates a distorted view, especially for those seated 
furthest from the camera. FIG. 12 shows an overhead view 
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of the typical videoconferencing arrangement, including 
conference table 300. camera and monitor 302, and confer- 
ence participants, shown as circles as 304. 

Using the PC-based monitoring system, it is possible to 
5 create a video conference that presents a much more natural 
viewing appearance. As shown in FIG. 13A. a multiple- 
camera, multiple-display unit 310 preferably is preferably 
located directly on the conference table 312. This reduces 
the camera-perspective-distorting effects just described, 
10 because conference members 314 may be seated in a more 
comfortable and convenient position. The resulting video 
image is aiso much more natural. Each controller remote site 
computer-display section 322 (as described herein below) 
simultaneously shows ail of the members participating in the 
15 remote conference, with an individual video window allo- 
cated for each participant, under control from the PC-based 
monitoring system operator. In practice, the remote site 
equipment operator will select one of the display operating 
modes as described in FIG. 15. depending on the number of 
subjects (camera views) and the capabilities of the remote 
site computer equipment. Optionally, an additional camera 
316 fitted with a wide-angle lens will provide an overall 
view of the conference room, in accordance with more 
traditional systems. 

To facilitate the aiming of the individual cameras for the 
participants, an arrangement such as that implemented at 
airport gates to assist the parking of airplanes (typically an 
"I" and an "O" formed of neon light tubes is utilized for 
3q airplane gate parking) optionally may be included. In this 
case, the two indicators are mounted so as to overlap only 
when viewed directly along the optical axis of the camera 
lens system. Thus, the camera position need only be adjusted 
so as to cause the indicators to overlap, thereby assuring that 
the camera correctly is aimed at the subject A number of 
microphones 318 are situated around the conference table in 
locations suitable for proper audio coverage of the partici- 
pants' speech. 

FIG. 13B shows a side-view depicting the physical layout 
of each of the four sides of the multiple-camera, multiple- 
display unit 310. The top unit 320 houses the individual 
cameras (2 to 8 units) and associated electronics, optionally 
including aiming provisions as described herein above. An 
LCD display screen 322 shows the various camera images of 

45 the conference participants. Speakers located at positions 
324 provide audio from the remote sites. 

As shown in FIG. 14. the multiple-camera/display unit 
shown in FIG. 13B preferably is modular in construction 
including removable side panels, in order to facilitate expan- 

50 sion along either horizontal axis so as to accommodate any 
size or shape conference table, with each section of the unit 
serving up to four conference participants. In the preferred 
embodiment, the entire unit preferably is constructed so as 
to provide a low profile, by utilizing slanted LCD display 

55 panels 330 to create a minimum obstruction for the local 
conference participants. If a low profile design is not 
required, the display panels optionally could be larger, and 
mounted vertically. Until color LCD panels become mere 
cost-effective, the display unit would be constructed with 

60 of the currently available small LCD projector systems. 
Each conference location may include a designated PC 
operator, however, the implementation of the network con- 
nection facilitates control of the system from a remote 
location. 

65 It should be noted that the implementation of the 
PC-based mocutonng system is not limited to these 
examples of security systems or videoconferencing. Many 
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alternative implementations, such as workplace, factory, 
production line, and process monitoring, would benefit from 
this system, and these alternative implementations should be 
considered to be within the scope of the invention. 
We claim: 



1. A video storage and display system, comprising: 

a plurality of video cameras, each outputring a signal 
representative of a video image; 

means to receive the signals from each camera and 

digitally compress the images; 
two forms of high-capacity storage media, one being 

randomly searchable while the other continues to store 

the digitally compressed image; and 
a computer configured to receive the digitally compressed 

images, the computer being interfaced to the following 

devices: 

a display screen. 

means to receive externally derived operator 
commands, and 

the high-capacity storage media, and wherein the com- 
puter is programmed to perform the following func- 
tions: 

display the digitally compressed images from the 
cameras in different windows on the display 
screen, each window being associated with an 
update rate and dimensions in pixels. 

vary the dimensions and the rate at which a particular 
image is updated in its window in accordance with 
one of the externally derived commands. 

store the digitally compressed images in the high- 
capacity storage medium, and 

vary the dimensions and the rate at which a particular 
image is stored in accordance with one of the 
externally derived commands. 

2. The video storage and display system of claim 1. 
further including means associated with the computer for 
controlling the operation of one or more of the video 
cameras. 

3. The video storage and display system of claim 1. 
wherein the means to digitally compress the image from a 
particular camera is disposed at the location of the camera. 

4. The video storage and display system of claim 1. 
wherein the means to digitally compress the image from a 
particular camera is disposed at the location of the computer. 

5. "The video storage and display system of claim 1. 
further including a separate computer associated with each 
camera, the computers being networked together over a 
common communication bus. enabling an operator situated 
at a particular computer to display the images gathered by 
other cameras in separate windows on that operator's dis- 
play screen. 

6. The video storage and display system of claim 1. 
wherein the high-capacity storage medium comprises a 
magnetic tape. 

7. The video storage and display system of claim 1. 
wherein the high-capacity storage medium comprises a 
magnetic disk. 

8. The method of simultaneously displaying and storing 
multiple video images, comprising the steps of: 

receiving video images from a plurality of sources; 
digitizing one or more of the images if not already in 
digital form: 
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displaying at least certain of the digitized images in 
separate windows on a display device, using a first, 
predetermined frame rate and resolution associated 
with each window; and 
simultaneously storing the displayed images using a 
second, predetermined frame rate and resolution asso- 
ciated with each image. 

9. The method of claim & further including the step of 
receiving a command to set the frame rate and resolution 
associated with the display and storage of a particular image. 

10. The method of claim 9, wherein the command is based 
upon an operator input. 

11. The method of claim 9, wherein the command is based 
l5 upon an external stimulus. 

12. The method of simultaneously displaying and storing 
multiple video images, comprising the steps of: 

receiving video images from a plurality of sources; 
digitizing one or more of the images if not already in 
20 digital form; 

displaying at least certain of the digitized images in 
separate windows on a display device, using a first set 
of temporal and spatial parameters associated with each 
image in each window; 

simultaneously storing the displayed images using a sec- 
ond set of temporal and spatial parameters associated 
with each image. 

13. The method of claim 12. the temporal parameters 
including frame rate. 

14. The method of claim 12. the spatial parameters 
- including image dimension in pixels. 

15. A video storage and display system, comprising: 

a plurality of video cameras, each outputting a signal 
35 representative of a video image: 

means to receive the signals from each camera and 

digitally compress the images; and 
a computer configured to receive the digitally compressed 
images, the computer being interfaced to the following 
40 devices: 

a display screen. 

means to receive externally derived operator com- 
mands including means for sensing a deviation from 
the normal- state image scene associated with at least 
45 one of the video cameras, the existence of the 

deviation being used as the basis for generating an 
externally derived command, and 
a high-capacity storage medium, and programmed to 
perform the following functions: 
50 display the digitally compressed images from the 

cameras in different windows on the display 
screen, each window being associated with an 
update rate and dimensions in pixels, 
vary the dimensions and the rate at which a particular 
55 image is updated in its window in accordance with 

one of the externally derived commands, 
store the digitally compressed images in the hiah- 

capaciry storage medium, and 
vary the dimensions and the rate at which a particular 
60 image is stored in accordance with one of the 

externally derived commands. 
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16 . (New) A video display system, c omprising: 

a plurality of video sources, each ou tputting a signal 

representative of a video image; 

means to receive the signals from each source and digital ly 
compress the images: and 

a computer configured to r eceive the digitally compressed 
images, the computer being inter faced to the following devices: 
a display screen, 

means to receive externally derived opera tor commands, 

and 

wherein the computer is programmed to perform the 

following functions: 

display the digitally compressed images from the 
cameras in different windows on the display screen, each 
window being associated with an update rate and 
dimensions in pixels, and 

vary the dimensions and the rate at which a 
particular image is updated in its window in a ccordance 
with one of the externally derived c ommands. 

17 . (New) A video sto rage system, comprising: 

a plurality of video sources . each putputting a signal 

representative of a video image: 

means to receive the signals from each source and digitally 
compress the images ; 

a high- capacity video storage medium; and 

a computer interfaced to th e following devices: 



an input to receive externally derived operator 
commands , and 

the high- capacit y storage media, and 
wherein the computer is programmed to perfor m the following 
functions : 

store the digitally compressed images in the high- 
capacity storage medium, and 

vary the dimensions and the rate at which a 

particular image is stored in accordance with one of the 
externally derived commands . 

18 . (New) The video storage system of claim 17 , further 

including two forms of high- capacity storage media, one being 

randomly searchable, and with the other being serial ly searchable. 

19 . (New) A method of displaying video images, comprising the 
steps of : 

receiving video images from a plurality of sources: 
digitizing one or more of the images if not already in digital 

f orm; 

displaying at least certain of the digitized images in 

separate windows on a display device f using a firs t r predetermined 
frame rate and a resolution associated wit h each window. 

20 . (New) A method of storing video images, comprising the 
steps of: 

receiving video images from a plurality of sources; 



Bi-.nr-i.na the images using a firs t- predetermined frame rate and 
resolution: and 

simultaneously storing the im a ges using a second predetermined 
frame rate and resolution. 
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ABSTRACT 



A PC- based system for monitoring and storing representa- 
tive images from video cameras may be utilized for security 
or other monitoring applications. Camera inputs from digital 
or analog sources are individually and independently digi- 
tized and displayed at a first set of image sizes, sampling 
rates, and frame rates, and may be stored in digital form on 
various recording media at a second set of image sizes, 
sampling rates, and frame rates, and these two sets of sizes 
and rates may or may not be identical. Provisions are 
included for adding detection or alarm systems which will 
automatically alter image size, sampling rate and/or frame 
rate of an individual input source, or activate other physical 
responses. In addition to security system monitoring, further 
applications of the invention are disclosed for process moni- 
toring in manufacmring environments and also for applica- 
tions in videoconferencing. 
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IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 

In re application of: Washino et al 

Serial No. : Group No. : 

Filed : Examiner : 

For: VIDEO MONITORING AND CONFERENCING SYSTEM 

REISSUE OF U.S. PATENT NO. 5,625,410 
ISSUED APRIL 29, 1997 

REISSUE DECLARATION AND POWER OF ATTORNEY 

Assistant Commissioner for Patents 
Washington, D.C. 20231 

Dear Sir: 

I, Kinya Washino, being duly sworn, do hereby depose and 
state as follows: 

1. I reside at 80 Hamilton Ave., Dumont, New Jersey. 

2. I am a joint inventor of the subject matter claimed 
in U.S. Patent No. 5,625,410. 

3. Barry Schwab is a joint inventor of the subject 
matter claimed in this patent. 

4. I believe that we are the original and first 
inventors of the subject matter which is being claimed with this 
re-issue application . 

5. I also believe that our issued patent is at least 
partly inoperative or invalid in that we claim less than we had the 
right to claim. 

6. In particular, certain of the claims as presently 
drafted include limitations which are unnecessary to distinguish 
the subject matter of our invention over the prior art. More 
specifically, the independent claims of our issued patent include 
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elements directed to the combination of display capabilities and 
high-capacity storage media when, in fact, such apparatus and 
methods of use may occur independent of one another, as set forth 
in the claims added on reissue. 

7 . This error in claiming was recently discovered and 
promptly acted upon. Over the course of the last several months, 
Mr. Schwab and I have been engaged in the formation of a business 
around the subject matter of our issued patents as they relate to 
multi- format audio and video production. 

8. In forming this new business, we have been in 
contact with various prospective partners and potential investors. 
In conjunction with these negotiations, we began a detailed and in- 
depth review of our relevant patents to ensure that we had claimed 
all that we were entitled to. 

9 . Following a series of telephone communications with 
our patent attorney, John G. Posa, we reached the conclusion that 
certain subject matter, as identified above, was not adequately 
represented in the claims, and that our issued patent was at least 
partially inoperative for failing to claim all that we had a right 
to claim in the patent. 

10. We therefore asked Mr. Posa to prepare this reissue 
application with the revised claims that we are submitting together 
with this declaration. 

11. These errors in claiming arose without any deceptive 
intention on the part of myself or my co- inventor, Mr. Barry 
Schwab . 
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12 . I have reviewed and understand the contents of the 
specification, including the claims, as they have been modified in 
accordance with this re-issue application, 

13 . I also acknowledge my duty to disclose to the Patent 
Office all information known to be material to the patentability of 
our invention, and have not identified any further references apart 
from those submitted or cited during the prosecution of our issued 
patent . 

14. I hereby appoint all of the attorneys of Gifford, 
Krass, Groh, Sprinkle, Anderson & Citkowski, P.C. as my legal 
representatives in this matter. All communications should be 
directed to John G. Posa, Esq. at the following address: 



GIFFORD, KRASS; GROH, SPRINKLE, 
ANDERSON £ CITKOWSKI, PC 
280 N. Old Woodward Ave. 
Suite 400 
Birmingham, MI 48009 
Telephone (734) 913-9300 



WHEREFORE , I hereby subscribe my name to the foregoing 
specification and claims, Declaration and Power of Attorney. 



Date: t i Kinya Washino 



THTQl P 



